Monitoring system and method, and program and recording medium used therewith

ABSTRACT

A monitoring system includes a picture capturing unit, a storage unit for storing original images composed of still images captured by said picture capturing unit, or a compressed images composed of compressed still images generated by compressing the still images, an image display unit, a designating unit for designating a region in an image displayed on the image display unit, and a control unit. The control unit controls said image display unit to display a panoramic entire image composed of the stored original images or compressed images, and updates only an image in the designated region in the entire image.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to a monitoring system and methodapplied to monitoring cameras, etc., and a program and a recordingmedium which are used with the system and method.

[0003] 2. Description of the Related Art

[0004] Monitoring systems have conventionally been used for monitoringsituations in wide areas. For example, monitoring systems have been usedfor sea surveillance, river surveillance, restricted area monitoring,wild animal behavior observation, etc. Since pictures of a wide areamust be captured, a video camera having a very large number of pixelshas been used. Accordingly, the system price is expensive, thus causinga problem in cost. Unlike this type of system, a method has beenproposed in which, by capturing still images while sequentially shiftingcapturing ranges of a camera, and connecting the images, an image in amonitored area is generated. In this method, an extremelyhigh-resolution image can be obtained as an entire image. Thus, when aportion of the entire image is obtained as an enlarged image, theenlarged image itself has a high resolution and is clear. Also, in theabove monitoring system, not only captured images can be monitored inreal time, but also images captured in the past can be monitored, withthe captured images stored in a memory. Moreover, by monitoring theimages captured in the past, and analyzing each image, the cause of, forexample, a particular event, such as an accident or a disaster, can beclarified.

[0005] Nevertheless, the monitoring systems of the related art have apoor operability in that it takes a long time to read and display imagesstored in the memory. This is because the amount of data per image isvery large since each image for image analysis must have a highresolution and clearness.

[0006] In addition, it is difficult for the above monitoring systems tomonitor a moving object such as a wild animal. This is because it isdifficult to shorten intervals at which an image displayed on a displayis updated since the data capacity of images is very large. In themonitoring system of the related art, not only captured images can bedisplayed, but also the captured images can be recorded on a recordingmedium, and can be transferred to a personal computer through a network.However, since the data capacity of the images is very large, it takes avery long time to perform a process for transferring the captured imagesfrom the monitoring system to the personal computer through the network.Also, a process for the monitoring system for recording the capturedimages on the recording medium, and a process for the personal computerfor reading images from the recording medium each require a very longtime.

SUMMARY OF THE INVENTION

[0007] It is an object of the present invention to provide a monitoringsystem and method in which intervals at which a displayed image isupdated are shortened, and a program and a recording medium which areused with the monitoring system and system.

[0008] It is another object of the present invention to provide amonitoring system and method in which the data capacity of images isreduced, and program and a recording medium which are used with themonitoring system and system.

[0009] According to a first aspect of the present invention, amonitoring system is provided which includes a picture capturing unitfor performing picture capturing, a storage unit for storing a pluralityof original images including a plurality of still images captured by thepicture capturing unit, or a plurality of compressed images includingcompressed still images generated by compressing the still images, animage display unit for performing image display, a designating unit fordesignating a region in an image displayed on the image display unit,and a control unit for controlling the image display unit to display apanoramic entire image including the stored original images orcompressed images, and updating an image in the designated region in theentire image by an image having a capturing time different from thecapturing time of each of images other the image in the designatedregion.

[0010] According to a second aspect of the present invention, amonitoring method is provided which includes a picture capturing stepfor performing picture capturing, a storage step for storing a pluralityof original images including a plurality of still images captured in thepicture capturing step, or compressed images including compressed stillimages generated by compressing the still images, a display step fordisplaying a panoramic entire image including the original images orcompressed images stored in the storage step, a designating step fordesignating a particular region in the entire image displayed in thedisplay step, and an updating step for updating the image in the regiondesignated in the designating step by an image having a capturing timedifferent from the capturing time of each of images other the image inthe designated region.

[0011] According to a third aspect of the present invention, a programis provided which causes a computer to execute a picture capturing stepfor performing picture capturing, a storage step for storing a pluralityof original images including a plurality of still images captured in thepicture capturing step, or compressed images including compressed stillimages generated by compressing the still images, a display step fordisplaying a panoramic entire image including the original images orcompressed images stored in the storage step, a designating step fordesignating a particular region in the entire image displayed in thedisplay step, and an updating step for updating the image in the regiondesignated in the designating step by an image having a capturing timedifferent from the capturing time of each of images other the image inthe designated region.

[0012] According to a fourth aspect of the present invention, arecording medium having a recorded program is provided. The programcauses a computer to execute a picture capturing step for performingpicture capturing, a storage step for storing a plurality of originalimages including a plurality of still images captured in the picturecapturing step, or compressed images including compressed still imagesgenerated by compressing the still images, a display step for displayinga panoramic entire image including the original images or compressedimages stored in the storage step, a designating step for designating aparticular region in the entire image displayed in the display step, andan updating step for updating the image in the region designated in thedesignating step by an image having a capturing time different from thecapturing time of each of images other the image in the designatedregion.

[0013] According to a fifth aspect of the present invention, amonitoring system is provided which includes a picture capturing unitfor performing picture capturing, a storage unit for storing image data,an image display unit for performing image display, a designating unitfor designating a region in the image displayed on the image displayunit, and a control unit for controlling the image display unit todisplay a panoramic entire image including a plurality of still imagescaptured by the picture capturing unit, and for controlling the picturecapturing unit in accordance with the designated region to capture stillimages in a position corresponding to the designated region anddisplaying, in the designated region, an image generated from the stillimages captured in the position.

[0014] According to a sixth aspect of the present invention, monitoringmethod is provided which includes a picture capturing step forperforming picture capturing, a display step for displaying a panoramicentire image including a plurality of still images captured in thepicture capturing step, a designating step for designating a particularregion in the entire image displayed in the display step, and a controlstep for controlling a picture capturing unit in accordance with theparticular region designated in the designating step to capture stillimages in a position corresponding to the designated region, anddisplaying, in the designated region, an image generated from the stillimages captured in the position.

[0015] According to a seventh aspect of the present invention, a programis provided which causes a computer to execute a picture capturing stepfor performing picture capturing, a display step for displaying apanoramic entire image including a plurality of still images captured inthe picture capturing step, a designating step for designating aparticular region in the entire image displayed in the display step, anda control step for controlling a picture capturing unit in accordancewith the particular region designated in the designating step to capturestill images in a position corresponding to the designated region, anddisplaying, in the designated region, an image generated from the stillimages captured in the position.

[0016] According to an eighth aspect of the present invention, arecording medium having a recorded program is provided. The programcauses a computer to execute a picture capturing step for performingpicture capturing, a display step for displaying a panoramic entireimage including a plurality of still images captured in the picturecapturing step, a designating step for designating a particular regionin the entire image displayed in the display step, and a control stepfor controlling a picture capturing unit in accordance with theparticular region designated in the designating step to capture stillimages in a position corresponding to the designated region, anddisplaying, in the designated region, an image generated from the stillimages captured in the position.

[0017] According to the present invention, a storage unit storesoriginal images including a plurality of still images captured by apicture capturing unit or compressed images including compressed stillimages generated by compressing the still images. A display unitdisplays a panoramic entire image generated from the stored originalimages or compressed images. A still image or compressed imagecorresponding to a region designated in the displayed entire image isread from the storage unit and is displayed. Thus, only an image in aparticular region in the entire image can be updated.

[0018] According to the present invention, an image display unitdisplays a panoramic entire image including a plurality of still imagescaptured by a picture capturing unit. The picture capturing unit iscontrolled in accordance with a region designated in the entire image bya designating unit. Still pictures are captured in a positioncorresponding to the designated region, and an image generated from thestill images captured in the position is displayed in the designatedregion. Therefore, only a desired region in the panoramic entire imagecan be updated. Thus, intervals at which the image is updated can beshortened.

[0019] According to the present invention, only an image in a particularregion in the entire image can be displayed. Therefore, the operabilityof analyzing images in a monitoring system can be improved.

[0020] In addition, only a desired region in a panoramic entire imagecan be updated. Thus, time intervals at which the image is updated canbe shortened.

BRIEF DESCRIPTION OF THE DRAWINGS

[0021]FIG. 1 is a block diagram showing a monitoring system according toan embodiment of the present invention;

[0022]FIG. 2 is a detailed block diagram showing a computer and a cameraunit in the monitoring system shown in FIG. 1;

[0023]FIG. 3 is a schematic illustration of an example of a screenhaving a GUI in the embodiment of the present invention;

[0024]FIG. 4 is a schematic illustration of an example of a selectionscreen in the embodiment of the present invention;

[0025]FIG. 5 is a schematic illustration of an example of a stored datadisplay screen in the embodiment of the present invention;

[0026]FIG. 6 is a perspective schematic illustration of a method forcreating an entire image in the embodiment of the present invention;

[0027]FIG. 7 is a schematic illustration of a range in which picturecapturing can be performed in the monitoring system shown in FIG. 1;

[0028]FIGS. 8A and 8B are schematic illustrations of a data managementmethod in the embodiment of the present invention;

[0029]FIG. 9 is a flowchart illustrating an operation of theentire-image display section 102 which captures frames of an entireimage according to one embodiment of the present invention;

[0030]FIG. 10 is a flowchart illustrating a process for playing backstored image data in the other embodiment of the present invention;

[0031]FIG. 11 is a flowchart illustrating a process of the monitoringsystem when it is in a multipoint view mode, according to one embodimentof the present invention;

[0032]FIG. 12 is a flowchart illustrating a process of the monitoringsystem when it is in a multipoint drive mode, according to oneembodiment of the present invention; and

[0033]FIG. 13 is a flowchart illustrating the process shown in FIG. 12of the monitoring system when it is in the multipoint drive mode.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Embodiments of the present invention are described below withreference to the accompanying drawings.

[0035]FIG. 1 shows the schematic structure of an embodiment of thepresent invention. A computer 1 connecting to a display 2 controlscamera units 3. FIG. 1 shows an example of a system in which onecomputer 1 controls two camera units 3, and a computer 1′ having adisplay 2′ controls another camera unit 3′. In this system, one computercan control a plurality of camera units.

[0036] Each camera unit 3 is formed by integrating a pan-tilter section4 and a camera section 5. The camera unit 3 is installed so as tocapture pictures of a remote target area. For example, the camerasection 5 includes a telephoto lens of 10 or 70 magnifications, etc, andcan capture pictures of a place located several tens of meters toseveral kilometers away.

[0037] The camera section 5 is, for example, a digital still camera thatcan switch on a shutter in synchronization with an external trigger. Itspicture-capture device, for example, a CCD has the number of pixels of astandard such as the VGA (Video Graphics Array) having a resolution of640 by 480 pixels, XGA (Extended Graphics Array) having a resolution of1024 by 768 pixels, or SXGA (Super Extended Graphics Array) having aresolution of 1280 by 1024 pixels. A VGA picture-capture device outputspicture data at a rate of 30 frames/second (fps). An XGA picture-capturedevice outputs picture data at a rate of 15 fps. An SXGA picture-capturedevice outputs picture data at a rate of 7.5 fps.

[0038] Video data is transmitted from the camera unit 3 to the computer1 by a bus 6. The bus 6 is used as a video-data transmission path and isused to transmit a control signal for the camera unit 3. The abovestructure similarly applies to the computer 1′ and the camera unit 3′.

[0039] Each of the computers 1 and 1′ stores, in its memory, video datafrom each of the camera units 3 and 3′. As is described later, each ofthe computers 1 and 1′ creates a graphical user interface (GUI) foroperation, and can control each of the camera units 3 and 3′ to capturepictures of a target area desired by a user. The captured pictures arecompressed by using, for example, JPEG (Joint Photographic ExpertsGroup).

[0040] The computers 1 and 1′ are connected to each other by a localarea network (LAN) 7. Another computer 8 is connected to the LAN 7. Thecomputer 8 has a display 9. The computer 8 receives picture data, etc.,from the computers 1 and 1′ through the LAN 7, stores the picture datain an archive 10, and processes the picture data. For example, thepicture data is used to perform face recognition, baggage recognition,car recognition, etc. The archive 10 is one that can store large amountsof data, such as a tape streamer.

[0041]FIG. 2 is a detailed block diagram showing the camera 1 and thecamera unit 3 in the above monitoring system. In the diagram in FIG. 2,the components of the computer 1 and the camera unit 3 are connected toa common controller bus 21.

[0042] The pan-tilter section 4 includes a pan portion 4 a and a tiltportion 4 b. The pan portion 4 a and the tilt portion 4 b each have astepping motor. In response to a control signal supplied from acontroller CPU (central processing unit) through the controller bus 21,the pan portion 4 a causes the camera unit 3 to pan, or the tilt portion4 b tilts the camera unit 3. On the pan-tilter section 4 is provided thecamera section 5. Here, the word “pan” means horizontally rotating thecamera unit 3, and the word “tilt” vertically rotating the camera unit3. By way of example, the maximum pan angle is set at 180 degrees, andthe maximum tilt angle is set at 50 degrees.

[0043] As is described later, the camera unit 3 is moved in its maximummovable range, and in the range of tilt angles of ±15 degrees and therange of pan angles of ±50 degrees. Whenever the center of picturecapturing is moved for an angle of view, the shutter is switched on tocapture still images (hereinafter referred to also as “frames”). In thecase of M (e.g., 8) vertical frames and N (e.g., 16) horizontal frames,a total of (M×N=8×16=) 128 frames are sequentially captured, and the 128frames are compressed and connected to one another to form an entireimage. Each frame is, for example, an XGA image having a resolution of1024 by 768 pixels. Thus, the 128 frames constitute an image composed ofapproximately one hundred million pixels (having (1024×16=) 16384horizontal pixels by (768×8=) 6144 vertical pixels) when overlappingportions are ignored. Capturing the 128 frames requires approximately 5seconds. Each overlapping portion has, for example, 16 vertical pixelsand 16 horizontal pixels.

[0044] The camera unit 3 has a digital camera structure, and includes alens unit 22, a focusing-zooming-iris control unit 23, and a capturingunit 24 (indicated by “CCD”). The focusing-zooming-iris control unit 23is controlled based on a control signal supplied from the controller CPU33 through the controller bus 21. The capturing unit 24 includes asolid-state image sensing device, for example, a charge-coupled device,and a camera-signal processing circuit. A digital picture signal fromthe capturing unit 24 is written into a buffer memory 26 through aninterface 25.

[0045] Output data from the buffer memory 26 is supplied to aJPEG-encoder/meta-data adder 27, and the picture data is converted intoJPEG data. JPEG is one of compression methods. Another compressionmethod may be used, or the picture data may not be compressed.

[0046] The camera unit 3 includes a global positioning system (GPS) 28for detecting the position of the camera unit 3. Since the camera unit 3includes the GPS 28, data of the installing location of the camera unit3 can be recorded, and the direction of the camera unit 3 can bedetected, thus enabling cooperative control of the directions of cameraunits. The GPS 28 is controlled based on a control signal supplied fromthe controller CPU 33 through the controller bus 21.

[0047] Output signal from the GPS 28 is supplied to a meta datagenerator 29. The meta data generator 29 generates positionalinformation (information such as latitude, longitude, direction, andaltitude) based on the result of positioning performed by the GPS 28,meta data (information such as time and parameters (such asmagnification, a focusing value, and an iris value) of the camera unit3), etc. The positional information and meta data are supplied to theJPEG-encoder/meta-data adder 27, and the supplied positional informationand meta data are added to the JPEG data.

[0048] The JPEG data having the added positional information and metadata is stored in the main memory 30 such as a hard disk, and issupplied to the graphic controller 31 and the image compressor 32. Inthis Specification, to store data in the main memory 30 is called “torecord data”, and to read data from the main memory 30 is called “toplay back data”. Also, to display an image being presently capturedwithout using the main memory 30 is called a “live mode”, and toplayback and display data recorded in the past is called a “view mode”.

[0049] The main memory 30 has a server function. For example, in thecase of using JPEG to compress an XGA image, the amount of data of oneframe is 100 kilobytes. Thus, 128 images have a data amount of 12.8megabytes. When the main memory 30 has a capacity of approximately 80gigabytes, it can store JPEG data per day. In the view mode, not onlydata stored in the main memory 30, but also older data stored in astorage unit such as the archive 10 can be played back.

[0050] JPEG data read from the main memory 30 is supplied to a graphiccontroller 31. An image compressor 32 generates a compressed image or athumbnail from JPEG data supplied from the JPEG-encoder/meta-data adder27 or JPEG data read from the main memory 30. For example, by decimatingvertical pixels and horizontal pixels, a panoramic entire image isformed. The image compressor 32 also performs compressing processing forforming a movable range image (described later). As described above, inthe case of XGA, data having approximately one hundred million pixels isprocessed by JPEG compression and the image compressor 32 to form apanoramic image having a resolution of 400 by 1000 pixels. The movablerange image is also a thumbnail, and is lower in resolution than theentire image.

[0051] The graphic controller 31 performs graphics processing thatperforms converting JPEG data into bitmap data and displaying a desiredimage on the screen of the display 2. Specifically, the display 2displays, on the screen, the movable range image, the entire image,selectable images, and a GUI such as buttons. Details of the displayedimages and the GUI are described later.

[0052] The graphic controller 31 detects a change in image by performingimage processing. The change in image is a change occurring comparedwith a reference image. For example, in the view mode, by performingcomparison with a previously stored reference image, a change in imageis detected. As a reference image, an image at a predetermined time onthe previous day is set, and between the reference image and an imagestored after the reference image is stored, a difference in pixel levelis detected. In a case in which the absolute value of the difference inpixel is equal to or greater than a predetermined value, the graphiccontroller 31 regards the case as occurrence of a change, and detectsthe change. Detection of a difference uses a method in which, for eachframe in spatially identical position between a reference image and animage to be compared therewith, differences in pixel level arecalculated. Instead of detecting differences concerning all the pixels,differences concerning typical pixels or decimated pixels may becalculated. Also, by using limitation to a predetermined color, achange, clarified when an attention is paid to an object in thepredetermined color, can be detected.

[0053] After the change is detected, the display 2 displays an alarmscreen on which, for example, a frame in which a change is detected canbe distinguished from another frame. Specifically, an alarmrepresentation can be displayed by a technique such as a brightnesschange, color change, or blinking. As the reference image, apredetermined image can be arbitrarily selected from among storedimages.

[0054] As described above, the controller CPU 33 connected to thecontroller bus 21 controls the camera unit 3, such as lens control(e.g., focusing, etc.), exposure control (e.g., control of diaphragm,gain, electronic shutter speed, etc.), white balance control, and imagequality control, and controls the pan portion 4 a and the tilt portion 4b.

[0055] A keyboard (K/B) 35 and a mouse 36 are connected to aninput/output (I/O) port 34. Also, a memory card 37 and a clock 38 areconnected to the I/O port 34. Positional information and JPEG data towhich meta data is added, which are stored in the main memory 30, can bewritten in the memory card 37. From the clock 38, time data is obtained.

[0056] In FIG. 2, components are connected to the controller bus 21.However, the camera unit 3 and the computer 1 may be installed in remoteplaces and both may be connected to each other by using a connectionsuch as IEEE1394 or USB (Universal Serial Bus). In this case, an opticalfiber is used as a physical transmission line. By using an opticalfiber, the camera unit 3 and the computer 1, which controls the cameraunit 3, can be installed, with several hundred meters to severalkilometers provided therebetween. In addition, both may be connected toeach other by a wireless LAN.

[0057]FIG. 3 shows an example of a GUI screen in this embodiment. Adisplay section, operation buttons, a display field, etc., which aredisplayed on the GUI screen, are described below with reference to FIG.3.

[0058] The GUI screen includes a movable-range-image display section101, an entire-image display section 102, and a selected mage displaysection 103 In the movable-range-image display section 101, the movablerange image is displayed. The movable range image is an image showingthe maximum range in which the camera unit 3 can perform picturecapturing, and consists of a plurality of frames. As described above,the maximum pan angle is 180 degrees and the maximum tilt angle is 50degrees, and a plurality of frames which are captured in the maximummovable range based on the above two maximum angles are processed togenerate a movable range image. For example, when the camera unit 3 isinstalled and starts picture capturing, the camera unit 3 is moved overthe maximum movable range, and an image composed of a plurality offrames is obtained. A thumbnail, generated by decimating the verticaland horizontal pixels of the obtained image, is used as the movablerange image.

[0059] In the movable-range-image display section 101, a position(hereinafter referred to as a “camera live position”) to which theoptical axis of the lens of the camera unit 3 is directed is indicatedby an intersection point of a line segment 101 a and a line segment 101b. By moving the line segments 101 a and 101 b, a desired position canbe designated in the movable range image, and the direction of picturecapturing can be controlled to move in the direction of the designatedposition. With the direction of the designated position used as a centeror a home position, (M×N) frames (still images) are captured in apredetermined movable range, and are stored or displayed. Not only bydesignating the line segments 101 a and 101 b, and but also by using apointer to designate an arbitrary position on the display screendisplayed in the movable-range-image display section 101 so that theoptical axis of the lens of the camera unit 3 is directed to a positioncorresponding to the designation, the camera unit 3 may be controlled.

[0060] In the entire-image display section 102, the panoramic entireimage is displayed. The panoramic entire image is an image generated byusing the image compressor 32 to compress JPEG data corresponding to theoriginal image captured by the camera unit 3. By viewing the displayedentire image, monitoring can be performed. As described above, when thechange in image is detected, an alarm screen appears in which, in theentire image displayed in the entire-image display section 102, a framefrom which the change is detected is displayed differently from theother frames.

[0061] In the selected mage display section 103, a selected image isdisplayed. The selected image is an enlarged image of a portion of theentire image. By displaying one frame of the original image inuncompressed form, an enlarged image is obtained. In addition, digitalsignal processing is used to enlarge an image.

[0062] An EXIT button 104 is used to switch off the power supply of themonitoring system. A Camera system OFF button 105 is used to switch offthe power supply of the camera unit 3.

[0063] A VIEW MODE button 106 is used to switch the mode of themonitoring system to the view mode. In the view mode, the entire imageand a partial image are displayed based on image data stored in the mainmemory 30 or in another server.

[0064] A LIVE MODE button 107 is used to switch the mode of themonitoring system to a live mode. In the live mode, the entire image anda partial image are displayed based on a frame which is being presentlycaptured by the camera unit 3.

[0065] A Compass display field 108 is used to display a compassindicating the direction in which the optical axis of the lens of thecamera unit 3 is directed. A GPS Data field 109 is used to display thelatitude, longitude, and altitude of the place at which the camera unit3 is installed, and the date and time of picture capturing. Pieces ofdata displayed in the fields 108 and 109 are those measured by a globalpositioning system (GPS) provided in the camera unit 3.

[0066] View offset buttons 110A, 110B, 110C, and 110D are used to adjustthe location of the selected frame. The View offset buttons 110A, 110B,110C, and 110D are respectively used to move, in upward, downward, left,and right directions, a frame which is selected, in the entire imagedisplayed in the entire-image display section 102, by the pointer. Theframes constituting the entire image are connected to one another sothat predetermined numbers of (e.g., 16) pixels of two adjacent framesoverlap each other. By moving one frame in the overlapping portion,consistency with an adjacent frame can be established, thus smoothingthe connection state.

[0067] A mode display field 129 is used to display mode information,alarm information, error information, etc. The mode information informsthe user of the mode of the monitoring system, and is specificallyinformation such as the live mode and the view mode. The alarminformation warns the user, and is displayed, for example, when amovement of the frame by the View offset button 110 reaches a movablelimit. The error information informs the user of an error occurring inthe monitoring system.

[0068] A Camera Control section 111 includes a ZOOM button 112, a FOCUSbutton 113, an IRIS button 114, a Camera Configuration button 115, and aWhite Balance button 116. The ZOOM button 112 is used to adjust zoomingof the camera unit 3. The FOCUS button 113 is used to adjust focusing ofthe camera unit 3. The IRIS button 114 is used to perform irisadjustment of the camera unit 3. The Camera Configuration button 115 isused to adjust the gamma characteristic, shutter speed, gaincharacteristic, etc., of the camera unit 3. The White Balance button 116is used to adjust the white balance of the camera unit 3. When thecamera unit 3 is in the view mode, the information displayed on theCamera Control section 111 may be omitted.

[0069] A SELECT button 117 is used to display a selection screen in theview mode. The selection screen is a picture in which a desired fieldfor playback and recording is specified by using the frames constitutingthe entire image.

[0070]FIG. 4 shows an example of the selection screen. On the selectionscreen, a grid indicating frame partitions is superimposed on the entireimage. For example, by using the pointer to designate a frame in adesired position on the selection screen, the designated frame isselected, and the brightness, resolution, contrast, etc., of thedesignated frame are changed in order to indicate that the designatedframe is selected.

[0071] A REC MODE selecting menu 118 is a pulldown menu for selecting arecording mode. The pulldown menu displays recording modes which arecombinations of image sizes for recording and recording methods (RUN andSINGLE). As the image size, any one of an entire image composed of (8 by16) frames, a partial image composed of (4 by 8) frames selected in theentire image, and a partial image composed of (2 by 4) frames selectedin the entire image can be selected. Each partial image is selected inthe position selected from the selection screen. In the recording methodRUN, captured images generated at predetermined periods (e.g., 5-secondperiods) are recorded. In the recording method SINGLE, the capturedimages are recorded only once. A combination of both modes can beselected as a recording mode.

[0072] A Stage Configuration button 119 (indicated by “Stage Config”) isused to fine-adjust stage-moving precision, etc. A message field 120 isused to display the connection status between the computer 1 for controland the camera unit 3 and the control status of a camera unit stage.When the computer 1 and the camera unit 3 are connected to each other,as FIG. 3 shows, the message “IMAGE SERVER CONNECT” is displayed in themessage field 120. When the camera unit stage is in a controllablestatus, the message “STAGE CONTROL ACTIVE” is displayed in the messagefield 120.

[0073] A REC button 121 is used to initiate image recording. By pressingthe REC button 121, recording in accordance with the selected recordingmode. Specifically, recording in accordance with a mode selected fromamong the recording modes RUN (8×16), RUN (4×8), RUN (2×4), SELECTSINGLE RUN (8×16), SELECT SINGLE RUN (4×8), and SELECT SINGLE RUN (2×4)is initiated.

[0074] A PLAY button 122 is used to play back image data stored in aserver (the main memory 30). Specifically, by pressing the PLAY button122, a stored data display screen is displayed. The stored data displayscreen displays information for identifying the stored image data. Theinformation is based on information described in a direction file.

[0075]FIG. 5 shows an example of the stored data display screen. Asshown in FIG. 5, the stored data display screen displays a minimizationbutton 161, a maximization button 162, a close button 163, a datedesignating field 164, a time designating field 165, an stored datadisplay field 165, a latest stored data display field 167, and an OKbutton 168, a cancel button 169, and a storage-unit change check button170.

[0076] The minimization button 161 is clicked on when the stored datadisplay screen is minimized to, for example, an icon. The maximizationbutton 162 is clicked on when the stored data display screen ismaximized for using the entire display area of the display screen. Theclose button 163 is clicked on when the stored data display screen isclosed.

[0077] In the date designating field 164, the date of stored data to bedisplayed in the entire-image display section 102 is designated. Forexample, by clicking on a button 164 a at the right end of the datedesignating field 164, the dates of stored data which can be displayedare displayed in a pulldown-menu form. From the displayed dates, adesired date may be selected.

[0078] In the time designating field 165, the time of a stored data itemto be displayed in the entire-image display section 102 is designated.For example, by clicking on a button 165 a at the right end of the timedesignating field 165, displayable times of stored data items aredisplayed in the form of a pulldown menu. From the displayed times, adesired time may be selected.

[0079] In the stored data display field 165, among the stored dataitems, an stored data item having the date and time designated by thedate designating field 164 and the time designating field 165 isdisplayed. In the latest stored data display field 167, among the storeddata items, the latest data item is displayed. Alternatively, amongstored data items having dates and times designated by the datedesignating field 164 and the time designating field 165, the lateststored data item may be displayed.

[0080] The Ok button 168 is clicked on when a desired stored data itemis designated. The cancel button 169 is clicked on when the stored datadisplay screen is closed. In the storage-unit change check button 170, acheck mark is input when the source of the stored data is changed fromthe storage unit of the server to, for example, a removably loadedsemiconductor memory (memory card).

[0081] Referring back to FIG. 3, a STOP button 123 is used to stop arecording or playback operation. The STOP button 123 may be displayed bypressing the REC button 121 or the PLAY button 122.

[0082] A Set Camera Center Position (Set Camera Center POS) button 125is used to designate the direction in which the camera unit 3 ispresently directed, as the center of an image of 8 by 16 frames.

[0083] A HOME button 124 is used to direct the optical axis of thecamera unit 3 to a home position by controlling the camera unit 3. Thehome position is a position in which the camera unit 3 is directed tothe leftest. LIVE/VIEW POSITION buttons 126 are used to move the cameraunit 3 to pan or tilt.

[0084] ZOOM buttons 127A and 127B are respectively used to enlarge andreduce the size of the selected image displayed in the selected magedisplay section 103. A MAX VIEW button 128 is used to display theselected image on a separate screen, for example, the entire-imagedisplay section 102.

[0085] Next, an example of a method for creating the entire imageaccording to an embodiment of the present invention is described belowwith reference to FIG. 6.

[0086] As shown in FIG. 6, the camera unit 3 is formed such that thecamera section 5 is mounted on the camera platform of the pan-tiltersection 4 so that the picture capturing direction from the home positioncan be changed. In FIG. 6, when being viewed from the camera unit 3, therows of (M by N) captured frames are sequentially numbered 1, 2, . . . ,M from top, and the columns of the (M by N) captured frames aresequentially numbered 1, 2, . . . , N from left. For example, a positionin which the frame having coordinates (1, 1) is captured is used as thehome position.

[0087] After the frame having coordinates (1, 1) is captured, the cameraunit 3 is moved to tilt downward, and the frame having coordinates(2, 1) is captured. Subsequently, the frames having pairs of coordinates(3, 1), . . . , (M, 1) are sequentially captured. Next, the frame whichis positioned at the top in the second column and which has coordinates(1, 2) is captured. The subsequent frames up to the frame havingcoordinates (M, N) are captured. As described above, each frame has a16-pixel portion overlapping another frame. The captured frames areprocessed, such as JPEG compression and storage to the main memory 30.

[0088] As described above, in a case in which each frame has an XGA form(1024 by 768 pixels), an image composed of 128 frames has approximatelyone hundred million pixels (having (1024×16=) 16384 horizontal pixels by(768×8=) 6144 vertical pixels) when overlapping portions are ignored. Inthe entire-image display section 102, a compressed image or thumbnailformed from the above image is displayed, and in the selected magedisplay section 103, an XGA image of, for example, one frame isdisplayed. Therefore, in the selected mage display section 103, an imagehaving a very high resolution can be displayed. Thus, also an unclearimage as an entire image can be displayed as a clear image.

[0089]FIG. 7 shows a range in which picture capturing can be performedby using one frame when the camera unit 3 is provided with a telephotolens of 75 magnifications. When a picture of a subject positioned 100meters away from the camera unit 3 is captured, one frame is used toenable capturing a picture of a range of 8.7 by 1.17 meters. Forexample, when XGA is used for the picture capturing device of the camerasection 5, a range of 0.87 by 1.17 centimeters of the subject can berepresented by approximately one pixel.

[0090] When a picture of a subject positioned 200 meters away from thecamera unit 3, one frame is used to enable capturing a picture of arange of 1.17 by 2.34 meters. For example, when XGA is used for thepicture capturing device of the camera section 5, a range of 1.74 by2.34 centimeters of the subject can be represented by one pixel.

[0091] When a picture of a subject positioned 500 meters away from thecamera unit 3, one frame is used to enable capturing a picture of arange of 4.36 by 5.84 meters. For example, when XGA is used for thepicture capturing device of the camera section 5, a range of 4.36 by5.84 centimeters of the subject can be represented by one pixel.

[0092] A data management method in the case of storing the acquired datain the archive 10, the main memory 30, etc., is described below withreference to FIGS. 8A and 8B.

[0093] As described above, images of M by N frames are captured atpredetermined time intervals, and are stored in compressed form. As FIG.8A shows, M rows and N columns are used to determine the positions ofthe frames. For example, the positional address (1, 1) specifies theframe at the top of the right end. Each frame has a set of such apositional address and recording-mode time information as a file name.The time information consists of a date (year, month, day) and a time(hours, minutes, seconds). Accordingly, the file name of the frame has a“year-month-day-hours-minutes-seconds-and-positional-address form.

[0094] As FIG. 8B shows, a direction file is determined corresponding tothe formation of an entire image from M by N frames. In the directionfile, by having data identical to the file name (i.e., a date and timeand a positional address) of a frame having the positional address (1,1), a set of the M by N frames is defined. The direction file also haspositional information and meta data corresponding to the set of theframes. The positional information and the meta data are generated bythe meta data generator 29. In other words, the direction file haspositional information such as latitude, longitude, direction, andaltitude, and meta data information such as the parameters (such asmagnification, a focusing value, and an iris value) of the camera unit3.

[0095]FIG. 9 is a flowchart illustrating an operation of theentire-image display section 102 which captures frames of an entireimage in an embodiment of the present invention.

[0096] When the LIVE MODE button 107 is designated by the pointer, andthe REC button 121 is designated by the pointer, the control algorithmof the flowchart is called.

[0097] In step S101, when, in the movable range image displayed in themovable-range-image display section 101, a capturing position isdesignated, it is determined in which point of the movable range imagethe entire image corresponds to, and coordinates for capturing theentire image are confirmed.

[0098] In step S102, a starting position for capturing the entire imageis calculated. Based on the result of the calculation, the pan portion 4a and the tilt portion 4 b of the camera unit 3 are controlled to movethe optical axis of the lens of the camera unit 3 to the startingposition of the capturing. Here, the starting position of the capturingis the position of the center of a first frame which is captured.

[0099] In step S103, by controlling the camera unit 3, specifically, thelens unit 22, the focusing-zooming-iris control unit 23, the capturingunit 24, etc., the frame is captured and is supplied as image data tothe computer 1.

[0100] In step S104, the image data supplied from the camera unit 3 isconverted into a predetermined image format, for example, JPEG.

[0101] In step S105, meta data and positional information are added tothe converted image data in the predetermined image format.

[0102] In step S106, the image data having the added meta data andpositional information is recorded in the main memory 30.

[0103] In step S107, the converted image data in the predetermined imageformat is displayed at a designated address in the entire-image displaysection 102, for example, the address (0, 0).

[0104] In step S108, a moving distance of the optical axis of the cameraunit 3 to the next frame is calculated.

[0105] In step S109, based on the moving distance calculated in stepS108, the pan portion 4 a and the tilt portion 4 b are controlled todirect the optical axis of the lens of the camera unit 3 to the centerof the next frame.

[0106] In step S110, the number of captured frames is calculated. Forexample, with the number of captured frames incremented whenever a frameis captured, the incremented number may be counted.

[0107] In step S111, it is determined whether or not the calculatednumber of the captured frames has reached a designated number of frames.If it is determined that the calculated number of the captured frameshas reached the designated number, control is transferred to step S112.If it is determined that the calculated number of the captured frameshas not reached the designated number yet, control is returned to stepS103. The designated number of frames is calculated beforehand inresponse to the mode selected in the REC MODE selecting menu 118.Specifically, when the mode RUN (8×16) is selected, the number of framesis 128. When the mode RUN (4×8) is selected, the number of frames is 32.When the mode RUN (2×4) is selected, the number of frames is 8.

[0108] In step S112, the distance from the present positioncorresponding to the optical axis of the lens of the camera unit 3 tothe starting portion for capturing in the entire-image display section102 is calculated.

[0109] In step S113, based on the distance calculated in step S112, thepan portion 4 a an the tilt portion 4 b are controlled to direct theoptical axis of the lens of the camera unit 3 to the center of the framewhich is used as the starting position for capturing.

[0110] In step S114, it is determined whether or not the number of timesthe entire-image display section 102 is updated has reached apre-designated number of times the entire-image display section 102 isupdated. Specifically, it is determined which of the selection mode andthe run mode in the REC MODE selecting menu 118 is selected.Accordingly, if it is determined that, in the REC MODE selecting menu118, the selection mode is selected, control is transferred to stepS115. If it is determined that, in the REC MODE selecting menu 118, therun mode is selected, control is transferred to step S117.

[0111] In other words, when, in the REC MODE selecting menu 118, theselection mode is selected, the number of times the entire-image displaysection 102 is updated is set to “1”. Accordingly, all the framesdisplayed in the entire-image display section 102 are captured once,recorded, and displayed. Thus, processing for re-capturing, recording,and displaying is not performed. Conversely, when, in the REC MODEselecting menu 118, the run mode is selected, the number of times theentire-image display section 102 is updated is set to “infinite”.Accordingly, processing for capturing, recording, and displaying isrepeatedly performed until capturing is terminated, that is, the STOPbutton 123 is pressed.

[0112] In step S115, the distance from the starting position forcapturing in the entire-image display section 102 to the central portionof the entire-image display section 102 is calculated. Based on theresult of the calculation, the pan portion 4 a and the tilt portion 4 bof the camera unit 3 are controlled to move the optical axis of the lensof the camera unit 3 to the central portion of the entire-image displaysection 102. The central portion of the entire-image display section 102is, for example, the central position of 8 by 16 frames.

[0113] In step S116, the operations of, for example, stepping motors ofthe pan portion 4 a and the tilt portion 4 b are stopped. Then, thecontrol algorithm of the flowchart ends.

[0114] Also, in step S117, it is determined whether or not the end ofcapturing is commanded. Specifically, it is determined whether or notthe STOP button 123 is designated by the pointer. If it is determinedthat the STOP button 123 is designated by the pointer, control istransferred to step S115. If it is determined that the STOP button 123is not designated by the pointer yet, control is returned to step S103.

[0115]FIG. 10 is a flowchart illustrating an operation for playing backstored image data in an embodiment of the present invention. By clickingon the PLAY button 122, the control algorithm of the flowchart isstarted.

[0116] In step S201, when the PLAY button 122 is clicked on, the storeddata display screen shown in FIG. 9 is displayed in the form of a popupwindow.

[0117] In step S202, it is determined whether or not a date isdesignated in the date designating field 164, and a time is designatedin the time designating field 165. If it is determined that the date isdesignated in the date designating field 164, and the time is designatedin the time designating field 165, control is transferred to step S203.If it is determined that both or either of the date in the datedesignating field 164 and the time in the time designating field 165 isnot designated yet, the control in step S202 is repeatedly performeduntil a date is designated in the date designating field 164 and a timeis designated in the time designating field 165.

[0118] In step S203, a movable range image and/or an entire image isdisplayed from stored data having the designated date and time.

[0119]FIG. 12 is a flowchart illustrating a process of the monitoringsystem when it is in a multipoint view mode in an embodiment of thepresent invention. In the multipoint view mode, only an image in adesignated position in the entire image is played back, tracing timeback.

[0120] In step S301, it is determined whether or not the SELECT button117 is pressed. If it is determined that the SELECT button 117 ispressed, control is transferred to step S302. If it is determined thatthe SELECT button 117 is not pressed, step 301 is repeatedly performed.

[0121] In step S302, a selection screen as described with reference toFIG. 4 is displayed. In step S303, it is determined whether or not aframe in the selection screen is selected by the pointer. If it isdetermined that the selection screen is selected, the process proceedsto step S304. If it is determined that no frame in the selectiond screenis not selected by the pointer, the process proceeds to step S306.

[0122] In step S304, display condition of the selected frame isswitched. For example, the brightness of the selected frame is switched.

[0123] In step S305, the coordinates of the selected frame areconfirmed.

[0124] In step S306, it is determined whether or not a CLOSE button 153on the selection screen is pressed. If it is determined that the CLOSEbutton 153 is pressed, the process proceeds to step S307. If it isdetermined that the CLOSE button 153 is not pressed, the process returnsto step S303.

[0125] In step S307, it is determined whether or not a PLAY button 122is pressed. If it is determined that the PLAY button 122 is pressed, theprocess proceeds to step S308. If it is determined that the PLAY button122 is not pressed, the process proceeds to step S307.

[0126] In step S308, a stored data display screen as described usingFIG. 5 is displayed. In step S309, it is determined whether or not theOK button 168 is pressed on the stored data display screen after adesired range of dates and time for playback is designated. If it isdetermined that the OK button 168 is pressed after the desired range ofdates and time is designated, the process proceeds to step S310. If itis determined that the OK button 168 is not pressed after the desiredrange of dates and time is designated, the process proceeds to stepS309.

[0127] In step S310, among image data items in the range of dates andtime designated in step S309, image data items corresponding to thecoordinates confirmed in step S305 are read from the main memory 30. Forexample, among the image data items in the range of dates and timedesignated in step S309, image data items corresponding to thecoordinates confirmed in step S305 are read from the main memory 30 inthe order of newer dates and time of picture capturing.

[0128] In step S311, in the position of the coordinates confirmed instep S305, images based on the image data items read in step S310 aredisplayed.

[0129] In step S312, it is determined whether or not all the image dataitems designated on the stored data display screen have been read. If itis determined that all the image data items designated on the storeddata display screen have not been read, the process proceeds to stepS310. If it is determined that all the image data items designated onthe stored data display screen have been read, the process ends.

[0130] The images in the frames other than the image in the framedesignated by the pointer are images having newer dates and time ofpicture capturing than the date and time of the image to be updated inthe designated frame, for example, images that are being captured by thecamera unit 3. In this case, only the image in the frame designated bythe pointer is updated, for example, tracing time back.

[0131] Also, the images in the frames other than the image in the framedesignated by the pointer may have older dates and time of picturecapturing than the date and time of the image in the designated frame.In this case, the image in the frame designated by the pointer isupdated, for example, going from the past to the present.

[0132] According to an embodiment of the present invention, thefollowing advantages can be obtained.

[0133] M by N frames captured by the camera unit 3 are compressed andstored in the main memory 30. An entire image composed the M by N framesstored in the main memory 30 is displayed on the display 2. Image datacorresponding to a frame designated in the displayed entire image by thepointer is read from the main memory 30 and is displayed in thedesignated frame. Accordingly, in the entire image, only a particularframe can be updated. Thus, the time required for reading the image datastored in the main memory 30 can be reduced. This enables an improvementin the operability of image analysis in the monitoring system.

[0134] Also, an image corresponding to the coordinates of a framedesignated in the entire image by the pointer can be read from the mainmemory 30 and can be played back. Therefore, the frame designated in theentire image by the pointer can be played back, tracing time back.

[0135] In addition, by viewing the movable range image, which indicatesa range in which the camera unit 3 can perform picture capturing, theuser can easily know in what range the location of the camera unit 3 cancapture pictures.

[0136] The embodiment of the present invention has been specificallydescribed. However, the present invention is not limited to theembodiment, and can be variously modified based on the technical idea ofthe present invention.

[0137] The above values described in the embodiment are simply examples.Different values may be used, if necessary.

[0138] Although the embodiment describes a configuration in which themonitoring system includes the JPEG-encoder/meta-data adder 27, theJPEG-encoder/meta-data adder 27 may be omitted. In other words, by usingM by N uncompressed original images, a panoramic entire image may bedisplayed on the display 2.

[0139] Although the embodiment describes a case in which an image in aframe designated by the pointer is updated (played back) from a newerdate and time of picture capturing, the image in the frame designated bythe pointer may be updated (played back) from an older date and time ofpicture capturing.

[0140] In the embodiment, the mode of the monitoring system may beswitched to the multipoint view mode by the following operation.

[0141] At first, a VIEW MODE button 106 is pressed by the pointer. Next,by using the pointer to press a SELECT button 117, a selection screen asdescribed using FIG. 4 is displayed. By designating a frame with thepointer on the selection screen, a desired region for view is selected.

[0142] Next, by pressing the PLAY button 122 with the pointer, a storeddata display screen is displayed. A range of dates and time for playbackare designated by using the pointer. Finally, the OK button 168 ispressed. This switches the mode of the monitoring system to themultipoint view mode.

[0143]FIGS. 12 and 13 are flowcharts illustrating a process of themonitoring system in the embodiment when the monitoring system is in amultipoint drive mode. In the multipoint drive mode, only an image in aposition designated in the entire image is updated.

[0144] In step S401, it is determined whether or not the SELECT button117 is pressed. If it is determined that the SELECT button 117 ispressed, the process proceeds to step S402. If it is determined that theSELECT button 117 is not pressed, step S401 is performed again.

[0145] In step S402, a selection screen as described with reference toFIG. 4 is displayed. In step S403, it is determined whether or not aframe in the selection screen is selected by the pointer. If it isdetermined that the frame in the selection screen is selected by thepointer, the process proceeds to step S404. If it is determined that theframe in the selection screen is not selected by the pointer, theprocess proceeds to step S406.

[0146] In step S404, display condition of the selected frame isswitched. For example, the brightness of the selected frame is switched.

[0147] In step S405, the coordinates of the selected frame areconfirmed.

[0148] In step S406, it is determined whether or not the CLOSE button153 is pressed. If it is determined that the CLOSE button 153 ispressed, the process proceeds to step S407. If it is determined that theCLOSE button 153 is not pressed, the process returns to step S403.

[0149] In step S407, it is determined whether or not the REC button 121is pressed. If it is determined that the REC button 121 is pressed, heprocess proceeds to step S408. If it is determined that the REC button121 is not pressed, step S407 is repeated again.

[0150] In step S408, the moving distance of the optical axis of the lensof the camera unit 3 to a frame from which capturing is initiated iscalculated.

[0151] In step S409, based on the moving distance calculated in stepS408, the pan portion 4 a and the tilt portion 4 b are controlled todirect the optical axis of the lens of the camera unit 3 to the centerof the next frame.

[0152] In step S410, by controlling the camera unit 3, specifically, thelens unit 22, the focusing-zooming-iris control unit 23, the capturingunit 24, etc., frames are captured and supplied as image data to thecomputer 1.

[0153] In step S411, image data supplied from the camera unit 3 isconverted into a predetermined image format, for example, JPEG.

[0154] In step S412, meta data and positional information are added tothe image data in the predetermined image format.

[0155] In step S413, the image data having the added meta data andpositional information is recorded in the main memory 30.

[0156] In step S414, the image data in the predetermined image format isdisplayed at a designated address in the entire-image display section102.

[0157] In step S415, the moving distance of the optical axis of the lensof the camera unit 3 to the next frame is calculated.

[0158] In step S416, based on the moving distance calculated in stepS415, the pan portion 4 a and the tilt portion 4 b are controlled todirect the optical axis of the lens of the camera unit 3 to the centerof the next frame.

[0159] In step S417, the number of captured frames is calculated. Forexample, with the number of captured frames incremented whenever a frameis captured, the incremented number may be counted.

[0160] In step S418, it is determined whether or not the number ofcaptured frames has reached a designated number of frames. If it isdetermined that the number of captured frames has reached the designatednumber of frames, the process proceeds to step S419. If it is determinedthat the number of captured frames has not reached the designated numberof frames yet, the process proceeds to step S410.

[0161] In step S419, the distance from the present positioncorresponding to the optical axis of the lens of the camera unit 3 tothe starting portion for capturing in the entire-image display section102 is calculated.

[0162] In step S420, based on the distance calculated in step S419, thepan portion 4 a an the tilt portion 4 b are controlled to direct theoptical axis of the lens of the camera unit 3 to the center of the framewhich is used as the starting position for capturing.

[0163] In step S421, it is determined whether or not the number of timesthe entire-image display section 102 is updated has reached apre-designated number of times the entire-image display section 102 isupdated. Specifically, it is determined which of the selection mode andthe run mode in the REC MODE selecting menu 118 is selected.Accordingly, if it is determined that, in the REC MODE selecting menu118, the selection mode is selected, the process proceeds to step S422.If it is determined that, in the REC MODE selecting menu 118, the runmode is selected, the process proceeds to step S424.

[0164] In other words, when, in the REC MODE selecting menu 118, theselection mode is selected, the number of times the entire-image displaysection 102 is updated is set to “1”. Accordingly, all the framesdisplayed in the entire-image display section 102 are captured once,recorded, and displayed. Thus, processing for re-capturing, recording,and displaying is not performed. Conversely, when, in the REC MODEselecting menu 118, the run mode is selected, the number of times theentire-image display section 102 is updated is set to “infinite”.Accordingly, processing for capturing, recording, and displaying isrepeatedly performed until capturing is terminated, that is, the STOPbutton 123 is pressed.

[0165] In step S422, the distance from the starting position forcapturing in the entire-image display section 102 to the central portionof the entire-image display section 102 is calculated. Based on theresult of the calculation, the pan portion 4 a and the tilt portion 4 bof the camera unit 3 are controlled to move the optical axis of the lensof the camera unit 3 to the central portion of the entire-image displaysection 102. The central portion of the entire-image display section 102is, for example, the central position of 8 by 16 frames.

[0166] In step S423, the operations of, for example, the stepping motorsof the pan portion 4 a and the tilt portion 4 b are stopped. Then, thecontrol algorithm of the flowchart ends.

[0167] Images in frames other the frame designated by the pointer are,for example, images having dates and time older than the date and timeof the image in the designated frame.

[0168] According to the embodiment, the following advantages can beobtained.

[0169] Compressed images formed by M by N still images captured by thecamera unit 3 are stored in the main memory 30. A panoramic entireimage, generated from the compressed images stored in the main memory30, is displayed on the display 2. By using the pointer to designate adesired frame to be updated in the entire image displayed on the display2, the image in the designated frame is captured by the camera unit 3.The captured image updates the desired frame. This can update only adesired frame in the panoramic entire image. Thus, time intervals atwhich image updating is performed can be shortened.

[0170] One embodiment of the present invention has been described.However, the present invention is not limited to the above embodiment,and can be variously modified based on the technical idea of the presentinvention.

[0171] For example, the values described in the above embodiment areonly examples. Different values may be used, if necessary.

[0172] In the above embodiment, the mode of the monitoring system may beswitched to the multipoint drive mode by the following operation.

[0173] At first, a VIEW MODE button 106 is pressed by the pointer. Next,by using the pointer to press the SELECT button 117, a selection screenas described using FIG. 4 is displayed. By designating a frame with thepointer on the selection screen, a desired region for the live mode isselected.

[0174] Next, the LIVE MODE button 107 is pressed by the pointer. In theREC MODE selecting menu 118, the selection mode is selected. Finally,the REC button 121 is pressed by the pointer. This switches the mode ofthe monitoring system to the multipoint drive mode.

What is claimed is:
 1. A monitoring system comprising: a picturecapturing unit for performing picture capturing; a storage unit forstoring a plurality of original images comprising a plurality of stillimages captured by said picture capturing unit, or a plurality ofcompressed images comprising compressed still images generated bycompressing the still images; an image display unit for performing imagedisplay; a designating unit for designating a region in an imagedisplayed on the image display unit; and a control unit for controllingsaid image display unit to display a panoramic entire image comprisingthe stored original images or compressed images, and updating an imagein the designated region in the entire image by an image having acapturing time different from the capturing time of each of images otherthe image in the designated region.
 2. A monitoring system according toclaim 1, wherein said control unit updates the image in the designatedregion in the entire image by a plurality of images having differentcapturing times.
 3. A monitoring system according to claim 1, whereinthe images other than the image in the designated region have capturingdates and times newer than the capturing date and time of the updatedimage in the designated region.
 4. A monitoring system according toclaim 3, wherein the images other than the image in the designatedregion correspond to images being captured by said picture capturingunit.
 5. A monitoring system according to claim 1, wherein the imagesother than the image in the designated region have capturing dates andtimes older than the capturing date and time of the updated image in thedesignated region.
 6. A monitoring method comprising: a picturecapturing step for performing picture capturing; a storage step forstoring a plurality of original images comprising a plurality of stillimages captured in said picture capturing step, or compressed imagescomprising compressed still images generated by compressing the stillimages; a display step for displaying a panoramic entire imagecomprising the original images or compressed images stored in saidstorage step; a designating step for designating a particular region inthe entire image displayed in said display step; and an updating stepfor updating the image in the region designated in said designating stepby an image having a capturing time different from the capturing time ofeach of images other the image in the designated region.
 7. A monitoringmethod according to claim 6, wherein, in said updating step, the imagein the region designated in the entire image in said designating step isupdated by a plurality of images having different capturing times.
 8. Amonitoring method according to claim 6, wherein the images other thanthe image in the designated region have capturing dates and times newerthan the capturing date and time of the updated image in the designatedregion.
 9. A monitoring method according to claim 8, wherein the imagesother than the image in the designated region correspond to images beingpresently captured.
 10. A monitoring method according to claim 6,wherein the images other than the image in the designated region havecapturing dates and times older than the capturing date and time of theupdated image in the designated region.
 11. A program for causing acomputer to execute: a picture capturing step for performing picturecapturing; a storage step for storing a plurality of original imagescomprising a plurality of still images captured in said picturecapturing step, or compressed images comprising compressed still imagesgenerated by compressing the still images; a display step for displayinga panoramic entire image comprising the original images or compressedimages stored in said storage step; a designating step for designating aparticular region in the entire image displayed in said display step;and an updating step for updating the image in the region designated insaid designating step by an image having a capturing time different fromthe capturing time of each of images other the image in the designatedregion.
 12. A program according to claim 11, wherein, in said updatingstep, the image in the region designated in the entire image in saiddesignating step is updated by a plurality of images having differentcapturing times.
 13. A program according to claim 11, wherein the imagesother than the image in the designated region have capturing dates andtimes newer than the capturing date and time of the updated image in thedesignated region.
 14. A program according to claim 13, wherein theimages other than the image in the designated region correspond toimages being presently captured.
 15. A program according to claim 11,wherein the images other than the image in the designated region havecapturing dates and times older than the capturing date and time of theupdated image in the designated region.
 16. A recording medium having arecorded program for causing a computer to execute: a picture capturingstep for performing picture capturing; a storage step for storing aplurality of original images comprising a plurality of still imagescaptured in said picture capturing step, or compressed images comprisingcompressed still images generated by compressing the still images; adisplay step for displaying a panoramic entire image comprising theoriginal images or compressed images stored in said storage step; adesignating step for designating a particular region in the entire imagedisplayed in said display step; and an updating step for updating theimage in the region designated in said designating step by an imagehaving a capturing time different from the capturing time of each ofimages other the image in the designated region.
 17. A recording mediumaccording to claim 16, wherein, in said updating step, the image in theregion designated in the entire image in said designating step isupdated by a plurality of images having different capturing times.
 18. Arecording medium according to claim 16, wherein the images other thanthe image in the designated region have capturing dates and times newerthan the capturing date and time of the updated image in the designatedregion.
 19. A recording medium according to claim 18, wherein the imagesother than the image in the designated region correspond to images beingpresently captured.
 20. A recording medium according to claim 16,wherein the images other than the image in the designated region havecapturing dates and times older than the capturing date and time of theupdated image in the designated region.
 21. A monitoring systemcomprising: a picture capturing unit for performing picture capturing; astorage unit for storing image data; an image display unit forperforming image display; a designating unit for designating a region inan image displayed on said image display unit; and a control unit forcontrolling said image display unit to display a panoramic entire imagecomprising a plurality of still images captured by said picturecapturing unit, and for controlling said picture capturing unit inaccordance with the designated region to capture still images in aposition corresponding to the designated region and displaying, in thedesignated region, an image generated from the still images captured inthe position.
 22. A monitoring system according to claim 21, wherein theimages other than the image in the designated region have capturingdates and times older than the capturing date and time of an image inthe designated region which is updated.
 23. A monitoring methodcomprising: a picture capturing step for performing picture capturing; adisplay step for displaying a panoramic entire image comprising aplurality of still images captured in said picture capturing step; adesignating step for designating a particular region in the entire imagedisplayed in said display step; and a control step for controlling apicture capturing unit in accordance with the particular regiondesignated in said designating step to capture still images in aposition corresponding to the designated region, and displaying, in thedesignated region, an image generated from the still images captured inthe position.
 24. A monitoring method according to claim 23, wherein theimages other than the image in the designated region have capturingdates and times older than the capturing date and time of an image inthe designated region which is updated.
 25. A program for causing acomputer to execute: a picture capturing step for performing picturecapturing; a display step for displaying a panoramic entire imagecomprising a plurality of still images captured in said picturecapturing step; a designating step for designating a particular regionin the entire image displayed in said display step; and a control stepfor controlling a picture capturing unit in accordance with theparticular region designated in said designating step to capture stillimages in a position corresponding to the designated region, anddisplaying, in the designated region, an image generated from the stillimages captured in the position.
 26. A program according to claim 25,wherein the images other than the image in the designated region havecapturing dates and times older than the capturing date and time of animage in the designated region which is updated.
 27. A recording mediumhaving a recorded program for causing a computer to execute: a picturecapturing step for performing picture capturing; a display step fordisplaying a panoramic entire image comprising a plurality of stillimages captured in said picture capturing step; a designating step fordesignating a particular region in the entire image displayed in saiddisplay step; and a control step for controlling a picture capturingunit in accordance with the particular region designated in saiddesignating step to capture still images in a position corresponding tothe designated region, and displaying, in the designated region, animage generated from the still images captured in the position.
 28. Arecording medium according to claim 27, wherein the images other thanthe image in the designated region have capturing dates and times olderthan the capturing date and time of an image in the designated regionwhich is updated.